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SECTION  I 
INTRODUCTION 

Providing  simulator  training  as  a surrogate  for  in-flight  training  is 
on  the  increase  in  the  military.  In  large  part,  this  has  come  about 
through  a developing  awareness  that  simulators  can  be  employed  to  advantage 
in  flight  training.  Several  specific  reasons  for  this  trend  can  be  identified. 

Substantive  advances  in  simulation  technology  are  reflected  in  the 
increased  sophistication  and  fidelity  of  visual  and  motion  systems 
and  in  the  dynamics  and  control  responsiveness  of  the  simulators. 

Concomitantly,  gains  in  the  strategies  of  training  have  shaped 
new  and  impressive  utilization  capabilities  for  flight  simulators. 

Substantial  economies  accrue  when  flight  simulators  are  employed 
efficiently  in  training  programs.  The  costs  of  aircraft 
operation  vis-a-vis  simulators  have  been  excessive  due  to  the 
recent  increases  in  fuel  and  maintenance  expenses.  It  is  estimated 
that  aircraft  hourly  operation  costs  can  be  10  times  those  of  the 
corresponding  simulators. 

Operational  considerations  generally  favor  simulators.  Foremost 
among  these  are  mechanical  reliability,  availability  of  training 
time,  compression  of  training  sequences,  and  compromises  due  to 
the  flight  environment;  e.g.,  safety,  weather,  and  airspace 
congestion. 

The  combination  of  engineering  sophistication  and  the  development  of 
systematic  student-centered  instructional  techniques  places  the  simulator 
quite  realistically  in  contention  as  a major  flight  training  medium  in 
today's  military  environment  (Smode,  1974).  To  these  should  be  added  the 
positive  feature  of  economic  advantage.  While  the  differentials  between 
simulator  and  aircraft  construction,  utilization,  and  amortization  are 
subject  to  various  interpretations,  the  evidence  generally  indicates  sig- 
nificantly lower  costs  for  training  when  the  simulator  is  used  efficiently 
in  conjunction  with  the  aircraft.  The  emphasis  on  fuel  economy,  as 
reflected  in  a recent  Department  of  Defense  (DOD)  directive1  calling  for 
a 25  percent  reduction  in  hours  flown  by  FY  1981,  has  intensified  the 
interest  in  the  costs  savings  associated  with  simulator  substitution 
practices. 

Unfortunately,  a useful  body  of  data  to  provide  guidelines  on  substitu- 
tion of  simulator  training  for  in-flight  training  does  not  exist.  Operating 
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practices  in  this  regard  are  also  sketchy.  The  evidence,  at  best,  indicates 
that  the  extent  of  and  confidence  in  substituting  simulator  training  for 
in-flight  training  are  a function  of  simulator  fidelity,  instructional 
strategy,  student  skill  level,  training  tasks,  aircraft  type,  and  training 
environment. 

It  is  not  the  intent  of  this  report  to  examine  the  many  facets  of  the 
aircraft/simulator  controversy  or  to  examine  the  generic  issue  of  transfer 
of  training.  The  latter  topic  has  been  well  addressed  in  a number  of 
recent  documents  (see,  for  example,  Smode,  Hall,  and  Meyer,  1966;  Hall, 
Parker,  and  Meyer,  1967;  Micheli,  1972;  Blaiwes,  Puig,  and  Regan,  1973; 

Caro,  1973;  Caro  and  Prophet,  1973;  Williges,  Roscoe,  and  Willi ges , 1973; 
Valverde,  1973;  Roscoe,  1974;  Hopkins,  1975;  Caro,  1976).  For  the  most 
part,  these  documents  organize  available  research  and  identify  trends  in 
data.  The  intent  here  is  straightforward  and  relatively  simple.  It  is  to 
determine  current  practices  in  substituting  simulator  training  for  in- 
flight training. 

PURPOSE 

This  report  provides  a sampling  of  current  practices  in  the  substitution 
of  simulator  training  for  in-flight  training.  This  work  is  a follow  on  to 
an  initial  quick  reaction  tasking  of  the  Training  Analysis  and  Evaluation 
Group  (TAEG)  by  the  Chief  of  Naval  Education  and  Training  (CNET)  wherein  an 
"immediate"  response  was  sought  on  current  military  users'  views  on  realistic 
substitution  ratios.  In  this  quick  inquiry  of  selected  military  units,  a 
wide  range  of  ratios  was  reported  which  suggested  that  in-flight  training 
was  arbitrarily  being  replaced  by  simulator  time.  Consequently,  the  present 
study  was  undertaken  to  provide  a detailed  examination  of  current  simulator 
substitution  practices.  The  goal  of  the  study  was  to  obtain  a useful  under- 
standing of  current  substitution  practices  and  to  determine  the  availability 
of  this  information.  Data  collection  was  initiated  in  January  1976  and 
essentially  completed  in  September  1976.  Information  was  obtained  by 
direct  solicitation  during  visits  to  operational  units  and  by  telephone 
interviews.  Also,  recent  literature,  unpublished  data,  and  other  informa- 
tion obtained  from  individuals  currently  involved  in  simulator  training  were 
used  in  the  review. 

No  attempt  is  made  to  provide  a methodological  or  theoretical  critique 
of  practices  in  individual  demonstrations.  However,  based  on  an  overview 
of  these  data,  a number  of  deficiencies  related  to  quality  of  the  data, 
interpretation  of  the  data,  and  methodologies  are  presented. 

ORGANIZATION  OF  THE  REPORT 

In  addition  to  this  introduction,  section  II  of  this  report  provides 
background  information  on  substitution  formulas  used  to  describe  the  data. 

The  two  substitution  formulas  utilized  in  the  report  are  percent  flight 
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syllabus  reduction  and  flight  substitution  ratio.  Explanatory  information 
is  provided  to  facilitate  the  understanding  of  data  treatment  in  subsequent 
sections.  Section  III  catalogs  data  on  the  substitution  practices  from  25 
demonstrations  into  3 major  classes  of  users:  general  aviation,  commercial, 

and  military.  Elaborations  of  the  curriculum,  training  practices,  or  equip- 
ment are  provided  only  when  such  additional  information  serves  to  aid  the 
interpretation  of  data  in  the  tables.  The  military  user  demonstration  data 
are  presented  by  type  of  airframe:  jet,  helicopter,  and  transport.  Section 

IV  summarizes  this  data  as  a function  of  the  factors  of  training  category, 
user  class,  aircraft  type,  student  experience,  simulator  visual  and  motion 
capabilities,  and  curriculum  features  such  as  the  use  of  special  syllabi  or 
part-task  trainers.  Section  V contains  a summary  and  discussion  of  major 
observations  and  deficiencies  gleaned  from  the  data. 
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SECTION  II 

SUBSTITUTION  FORMULA 

This  section  contains  an  explication  of  the  formulas  which  are 
currently  used  to  describe  the  relationship  between  simulator  substitution 
hours  and  in-flight  hours.  They  are  used  to  summarize  and  compare  the 
demonstration  data  provided  in  this  report.  Therefore,  familiarity  with 
the  assumptions  underlying  these  formulas  and  the  associated  computational 
methods  is  required  to  facilitate  understanding  the  information  presented 
in  sections  III  and  IV. 

FORMULAS 

Three  types  of  computational  formulas  in  current  use  are  presented 
in  table  1.  The  first  formula  (Percent  Flight  Syllabus  Reduction)  expresses 
the  overall  ability  of  the  simulator  to  reduce  the  amount  of  in-flight 
training  in  the  syllabus. 


TABLE  1. 

WAYS  OF  MEASURING  TRANSFER 

Computation 

Formula 

Percent  Flight  Syllabus 
Reduction 

Original  Flight  Hours  - New  Flight  Hours  v lnn 
Original  Flight  Hours  ' ' 

Flight  Substitution* 

New  Simulator  Hours  - Original  Simulator  Hours 

Ratio  (FSR) 

i 

Original  Flight  Hours  - New  Flight  Hours 

Transfer  Effectiveness* 
Ratio  (TER) 

Original  Flight  Hours  - New  Flight  Hours 
New  Simulator  Hours** 

* Although  these  values  are  defined  as  a ratio,  in  subsequent  sections 
of  the  report,  only  the  quotient  of  this  calculation  is  reported. 


**  Some  authors  use  the  new  simulator  hours  minus  original  simulator 
hours  as  the  denominator  where  the  original  group  received  some 
simulator  training  time  (i.e.,  when  there  was  no  flight-only  group). 

The  larger  the  positive  value  of  syllabus  reduction,  the  more 
effective  the  simulation.  Syllabus  reduction  can  also  be  negative  when 
the  simulator  group  received  more  flight  training  than  the  group(s) 
receiving  none  or  smaller  amounts  of  simulator  training.  The  percent 
flight  syllabus  reduction  is  sometimes  simply  referred  to  as  percent 
savings. 
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The  Flight  Substitution  Ratio  (FSR),  the  second  formula  in  table  1, 
refers  to  the  ratio  of  increase  in  simulator  hours  to  the  decrease  in 
flight  training  hours.  The  FSR  is  the  rate  at  which  flight  time  is  being 
replaced  by  the  simulator.  Thus  this  term  reflects  efficiency  of  the  device 
The  smaller  the  value  of  a positive  FSR,  the  more  effective  the  substitution 
A negative  FSR  may  be  obtained  when  a device  is  used  more  effectively  and 
results  in  decreases  in  both  simulator  and  in-flight  hours.  A negative  FSR 
may  also  occur  when  increased  simulator  hours  are  associated  with  an 
increase  in  flight  hours. 

The  third  formula  in  table  1,  Transfer  Effectiveness  Ratio  (TER), 
expresses  transfer  as  a ratio  of  flight  hours  saved  to  the  time  spent  in 
the  simulator.  This  ratio  is  essentially  the  reciprocal  of  the  FSR.  There- 
fore, only  FSR  is  reported  in  the  tables  in  this  report.  The  concept  of 
TER  is  discussed  in  detail  in  Roscoe  (1971). 

In  accounting  for  training  time  in  the  calculation  of  substitution 
values,  two  special  conventions  were  employed.  First,  the  inclusion  of 
simulator  time  for  the  original  group  in  the  calculation  of  substitution 
ratios  was  based  on  a judgment  of  the  effectiveness  of  this  time.  This 
judgment  was  made  in  all  cases,  but  only  in  one  case  did  it  lead  to  the 
exclusion  of  the  original  group's  simulator  time. 

The  second  convention  dealt  with  the  calculation  of  substitution 
of  values  in  multi  piloted  simulators  such  as  those  used  by  the  airlines 
and  some  military  environments.  In  these  situations,  students  normally 
occupy  both  pilot  and  copilot  positions.  One  student  occupies  the 
"training"  position  while  the  second  student  performs  the  role  of  the  other 
crew  position.  During  in-flight  instruction,  students  occupy  only  one  of 
these  positions.  Total  time  in  both  simulator  positions  was  used  in  the 
substitution  calculations.  However,  the  relevance  of  time  spent  in  the 
nontraining  position  might  be  questioned.  This  is  especially  true  for 
military  students  since  they  are  generally  not  tested  on  items  learned  in 
that  position. 
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SECTION  III 

CATALOG  OF  SIMULATION  SUBSTITUTION  PRACTICES 

Data  from  the  three  major  classes  of  aircraft  simulation  users  are 
cataloged  in  this  section.  Organizing  the  results  of  the  demonstrations 
according  to  the  classes  of  (1)  General  Aviation,  (2)  Commercial,  and 
(3)  Military  users  permits  both  interclass  and  intraclass  comparisons. 

The  term  "demonstration"  is  used  in  this  report  to  refer  to  comparisons 
of  two  training  programs  for  a particular  type  aircraft.  Typically  this 
involves  comparing  in-flight  training  hours  before  and  after  the  installa- 
tion of  a new  simulator,  or  the  increased  use  of  an  existing  device. 

The  tables  in  this  section  all  follow  the  same  format.  First,  the 
organizational  environment,  the  type  of  tasks  and  students  involved,  and 
the  aircraft  and  simulator  utilized  are  described.  For  all  demonstrations, 
the  information  provided  herein  is  as  detailed  as  that  reported  in  the 
primary  sources.  Second,  the  raw  training  times  data  are  presented  in 
a form  amenable  to  the  use  of  the  computational  formulas  described  in 
section  II.  Finally,  simulator  substitution  results  are  reported  in  the 
form  of  percent  flight  syllabus  reduction  (effectiveness)  and  flight 
substitution  ratios  (efficiency).  The  discussion  which  follows  amplifies 
information  depicted  in  various  tables  in  this  section.  The  reader  may 
wish  to  refer  to  the  appropriate  tables  for  the  details. 

GENERAL  AVIATION 

These  data  are  based  on  demonstrations  which  typically  use  private 
pilots  with  relatively  limited  flying  experience,  unsophisticated 
simulators,  and  light  aircraft.  They  are  examined  here  because  of  their 
experimental  rigor  and  innovativeness  in  research  methodology. 

As  expected,  demonstrations  1 and  2 in_table  2 indicate  that  a 
current  generation  aviation  trainer  (GAT-1 )d  having  a higher  fidelity  to 
the  modern  aircraft  achieved  better  substitution  results  than  a post- 
World  War  II  vintage  device  (AN-T-18).  The  large  differences  in  results 
of  demonstrations  2 and  3 were  due  presumably  to  differences  in  the 
instructional  strategies  and  controls.  Consistent  with  general  trends, 
the  instrument  training  tasks  in  demonstration  4 produced  better  substi- 
tution results  than  basic  aircraft  handling  tasks  in  demonstration  3. 


The  GAT  series  are  general  aviation  trainers.  The  GAT-1  simulates  a 
single-engine  light  aircraft;  the  GAT-2,  a piston-powered  light  twin. 
Reference  to  general  aviation  trainers  and  GAT,  a registered  trademark 
of  the  Link  Division  of  Singer,  Inc.,  does  not  constitute  an  official 
endorsement  or  approval  by  the  Navy  Department  of  a commercial  product 
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COMMERCIAL 

These  data  are  based  on  demonstrations  which  typically  use  highly 
experienced  pilots,  sophisticated  simulators,  and  jet  transport  category 
aircraft.  They  are  included  here  because  they  represent  a comparatively 
high  level  of  refinement  in  simulator  utilization.  The  data  obtained 
from  these  sources  are  also  characterized  by  a high  degree  of  reliability. 
Only  a restricted  sample  of  commercial  demonstrations  is  presented  here 
in  the  belief  that  they  are  representative  of  industry  practice. 
Demonstrations  5 and  6 in  table  3 reveal  the  successful  outcome  of 
airline  substitution  practices  with  FSRs  of  1.1  and  0.9  respectively  and 
90+  percent  syllabus  reductions.  In  contrast  to  these  demonstrations, 
which  involved  jet  transport  captain  transition  training,  demonstration 
7 utilized  corporate  pilots.  The  latter  demonstration  achieved  a lesser 
degree  of  substitution. 

MILITARY 

These  data  are  based  on  demonstrations  drawn  from  all  branches  of 
military  service.  Involved  are  military  pilots,  a wider  range  in  the 
sophistication  of  training  devices,  and  a variety  of  military  aircraft. 

The  sample  of  training  demonstrations  presented  here  are  representative 
of  current  military  practice.  Most  of  the  demonstrations  were  done 
under  the  cognizance  of  organizations  which  specialize  in  training 
system  development  and  evaluation.  However,  data  were  generated  from 
several  demonstrations  which  were  subject  to  operational  expediencies 
and  not  to  formal  monitoring  and/or  control.  This  discussion  of  military 
practice  is  organized  according  to  three  basic  categories  of  aircraft: 
helicopters,  jets  (center-line  thrust  fighter- type  trainers),  and  trans- 
ports (noncenter-line  thrust  piston,  turbo-prop,  and  jet  powered). 

HELICOPTERS.  These  substitution  practices  refer  to  demonstrations  8 
through  12  in  table  4.  Demonstration  8 employed  the  Army  Synthetic 
Flight  Training  System  (2B24),  a high-fidelity  device.  This  demonstration 
achieved  substitution  levels  approaching  those  obtained  by  the  commercial 
airlines.  The  following  factors  were  thought  to  enhance  the  success  of 
this  demonstration:  (1)  the  training  primarily  involved  instrument  proce- 

dures which  have  been  generally  shown  to  have  high  transfer,  (2)  the 
program  employed  a curriculum  designed  to  exploit  the  simulator's  unique 
capabilities.  In  addition  it  should  be  noted  that  the  previously  used 
device  (1-CA-l)  provided  negligible  transfer  to  the  helicopter  (Isley, 
Caro,  and  Jolley,  1968).  Therefore,  1-CA-l  time  was  ignored  in  calcu- 
lating the  FSR  presented  here. 

Demonstrations  9,  10,  and  11  show  a wide  variation  in  substitution 
and  considerably  less  success  than  demonstration  8.  In  demonstrations 
9 and  10  dramatic  differences  in  savings  were  obtained  even  though  the 
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TABLE  4.  MILITARY  DEMONSTRATIONS:  HELICOPTERS 
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training  curriculum,  simulator  (VCTS),  and  airframe  (H-52)  were  the  same 
with  the  only  difference  being  the  previous  flight  experience  of  the 
students. 

Demonstration  12  achieved  a moderate  degree  of  substitution.  In 
this  demonstration  the  Air  Force  obtained  a better  substitution  ratio 
than  did  the  Coast  Guard  in  demonstration  11  for  the  same  airframe  (H-3). 

In  both  demonstrations  a new  curriculum  was  developed  emphasizing  the 
simulator.  These  curriculums  included  specific  training  objectives  with 
self-paced,  proficiency-based,  advancement  for  students. 

JETS.  Demonstrations  13  through  18  in  table  5 describe  substitution 
practices  in  Undergraduate  Pilot  Training  (UPT)  utilizing  fighter-type 
jet  trainers.  Three  U.S.  Air  Force  demonstrations  (13,  14,  and  15)  were 
included  in  this  description  of  current  practices  evert  though  they 
employ  research-oriented  devices  (T-4G  and  the  Advanced  Simulator  for 
Undergraduate  Pilot  Training  (ASUPT)).  They  are  included  here  because 
the  results  of  these  well-conducted  studies  have  influenced  the  design 
of  an  operational  training  device,  the  Instrument  Flight  Simulator 
(IFS).  The  latter  could  be  viewed  as  an  operational  version  of  the 
research  devices  used  in  these  studies. 

The  results  of  the  two  T-37/T-4G  demonstrations  (13  and  14)  were 
consistent  with  the  general  trend  of  larger  percentage  savings  for  instru- 
ment vs.  contact  tasks.  The  negative  FSR  obtained  in  the  instrument  course 
was  due  to  the  unusual  circumstances  of  a simultaneous  decrease  in  both 
simulator  and  flight  time.  These  two  studies  employed  a simulator  with 
modest  visual  and  motion  systems.  A third  U.S.  Air  Force  T-37  demonstra- 
tion (15)  utilized  the  ASUPT.  This  is  a new  and  highly  sophisticated 
simulator  equipped  with  wide  angle  visual  and  large  amplitude  (six  degrees 
of  freedom)  motion  system,  G-seat,  and  G-suit.  This  demonstration 
incorporated  both  the  contact  and  instrument  tasks  of  the  U.S.  Air  Force 
basic  jet  curriculum.  The  resultant  savings  for  the  program  achieved  a 
level  between  the  values  reported  in  previous  T-37  demonstrations  (13 
and  14).  All  three  T-37  studies  used  a new  syllabus  designed  to  make 
maximum  use  of  new  simulator  capabilities.  Although  the  programs  employed 
self-paced,  proficiency-based,  advancement,  their  use  was  restricted  to 
"new  simulator"  groups.  This  differential  treatment  may  have  increased 
the  substitution  values  achieved  by  the  new  simulator  groups  over  respective 
"original  simulator"  groups. 

The  TA-4/2F90  aircraft  and  flight  simulator  combination  is  used  in 
several  stages  of  U.S.  Navy  and  U.S.  Marine  Corps  advanced  jet  training. 
Table  5 presents  three  demonstrations  from  this  program  (16,  17,  and 
18)  involving  instrument  training.  Increased  simulation  use  in  dem- 
onstrations 16  and  17  achieved  approximately  50  percent  flight  syllabus 
reduction  over  the  traditional  syllabus  groups.  This  figure  was  con- 
sistent with  that  obtained  in  the  U.S.  Air  Force  jet  instruments 
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demonstration  (14)  reported  earlier.  However,  simulator  efficiencies 
(FSRs)  were  markedly  better  in  the  Navy  studies. 

In  contrast  to  the  usual  practices  of  increasing  simulator  time, 
demonstration  18  attempted  to  determine  the  implications  of  total  elimina- 
tion of  simulator  time.  A comparison  of  the  flight-only  group  with  a 
previous  traditional-syl labus  group  indicated  that  the  former  required 
only  0.5  hours  more  flight  time.  This  resulted  in  an  FSR  of  42.0  which 
was  several  orders  of  magnitude  higher  than  those  obtained  in  demonstra- 
tions 16  and  17. 

TRANSPORTS.  A third  major  category  of  military  substitution  practice 
provided  data  that  were  based  on  demonstrations  which  involved  military 
pilots,  a range  of  simulator  capabilities,  and  multi  engine  fixed-wing 
aircraft. 

Demonstrations  19  and  20  in  table  6 involve  the  use  of  the  Navy  P-3 
aircraft  and  the  2F69  simulator.  The  purpose  of  demonstration  19  was  to 
improve  simulation  utilization  through  modification  of  syllabus  and 
instructional  techniques.  The  data  from  demonstration  20,  on  the  other 
hand,  were  generated  from  a situation  where  the  simulator  time  was 
available  from  previous  training;  the  flight-only  group  information  was 
the  consequence  of  a class  being  trained  during  a squadron  relocation 
which  temporarily  eliminated  use  of  the  simulator.  The  formal  effort 
(19)  to  explore  simulator  substitution  resulted  in  far  better  efficiency 
( FSR)  and  effectiveness  (syllabus  reduction)  than  the  more  informally 
generated  data  of  the  companion  demonstration  (20). 

Demonstration  21  provides  data  on  Device  2F87F,  P-3C  Operational 
Flight  Trainer.  The  study  was  concerned  with  an  assessment  of  the 
effectiveness  of  the  device  as  a substitute  for  the  current  analog 
operational  flight  trainer  used  in  combination  with  the  P-3  aircraft. 

The  2F87F  high-fidelity  digital  device  is  equipped  with  a six  degrees  of 
freedom  motion  and  a narrow  angle  television  model-board  visual  system. 

The  device  when  used  in  a block  training  regime  for  the  familiarization/ 
instrument  phase  of  transition  training  was  an  effective  substitute  for 
9.0  hours  of  training  in  the  older  analog  device  and  6.4  hours  of  in- 
flight training  in  the  P-3  aircraft.  This  represents  a 43  percent 
reduction  of  aircraft  training  time  and  provided  an  FSR  of  2.3.’  A 
significant  finding  of  this  study  was  the  reported  transfer  of  training 
for  the  landing  task  (a  reduction  of  16  landings  required  for  qualifica- 
tion). No  previous  studies  are  known  to  have  reported  transfer  of 
training  from  the  simulator  to  the  aircraft  for  this  task. 


The  relatively  high  FSR  results  from  the  addition  of  approximately  12 
hours  training  given  each  pilot  in  copilot  tasks  not  trained  or  checked 
in  the  aircraft. 
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Demonstration  22  contains  data  obtained  from  a comparison  of  two 
U.S.  Navy  squadrons  which  fly  the  same  basic  aircraft.  One  squadron 
employed  a flight  simulator  while  the  second  relied  on  a Cockpit  Pro- 
cedures Trainer  (CPT).  The  unit  wi thout  the  simulator  (2F65)  required 
6.5  hours  less  flight  time  to  transition  pilots.  No  significant  differ- 
ence in  piloting  tasks,  syllabi,  or  training  requirements  which  could 
have  contributed  to  this  finding  was  evident.  However,  the  2F65  simulator 
was  not  state-of-the-art,  lacked  a visual  system,  and  had  only  a limited 
motion  system. 

Two  U.S.  Air  Force  demonstrations  (23  and  24)  involved  heavy  trans- 
ports and  state-of-the-art  simulators  (T-19  and  T-37A).  Similar  to  the 
previous  Navy  P-3  aircraft  demonstrations  (19  and  20),  the  formally 
supervised  demonstration  (23)  achieved  substitution  superior  to  a less 
formal  demonstration  (24).  In  particular,  simulator  efficiency  (FSR) 
was  better  for  the  formally  supervised  demonstrations. 

The  final  demonstration  (25)  differed  from  the  other  transport 
studies  in  that  it  involved  a light  twin-engine  transport  (T-42)  and 
U.S.  Army  undergraduate  pilots.  The  substitution  values  obtained  in 
this  demonstration  were  superior  to  most  other  military  transport  programs 
cited  in  table  6.  The  simulator  group  in  this  demonstration  used  a 
twin-engine  General  Aviation  Trainer  (GAT-2)  while  the  comparison  group 
utilized  the  lower  fidelity  generalized  instrument  trainer  (2B12A). 
However,  this  significant  savings  was  obtained  only  after  a syllabus 
modification  based  on  the  new  device's  capabilities  was  introduced. 
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SECTION  IV 

SYNOPSIS  AND  FACTOR  COMPARISONS 

This  section  summarizes  the  demonstration  data  as  a function  of 
training  category,  user  class,  type  aircraft,  student  experience, 
simulator  capabilities,  and  curriculum  features.  Initially,  a tabular 
format  is  used  to  provide  a synopsis  of  the  factors  indigenous  to  each 
of  the  demonstrations.  This  is  followed  by  a series  of  tables  in  which 
comparisons  of  effectiveness  (percent  flight  syllabus  reduction)  and 
efficiency  (flight  substitution  ratio)  associated  with  various  factors 
in  the  demonstrations  are  presented. 

SYNOPSIS 

Table  7 summarizes  the  demonstrations  in  terms  of  their  significant 
situational  factors.  The  table  also  presents  data  on  the  percent  flight 
syllabus  reduction  and  the  flight  substitution  ratio  for  each  demonstra- 
tion. The  factors  form  the  bases  of  substitution  effectiveness  and 
efficiency  comparison  in  subsequent  parts  of  this  section.  Although 
additional  factors  could  have  been  examined,  these  six  factors  were  of 
primary  interest  in  the  present  effort.  Only  generalized  factors  have 
been  employed  because  more  specific  identifications  would  render  compari- 
sons impossible  due  to  paucities  of  data  at  finer  levels  of  analysis. 

Even  with  these  limitations,  there  are  1,296  potential  combinations  of 
the  parameters  in  table  7.  The  1 ,296 ^combinations  are  computed  as 
follows:  3 training  categories  X 3 user  classes  X 3 types  of  aircraft  X 

3 student  experience  levels  X 4 simulation  capabilities  X 4 curriculum 
features.  This  complexity  suggests  the  high  degree  of  caution  required 
in  interpreting  the  data. 

COMPARISONS  OF  EFFICIENCY  AND  EFFECTIVENESS 

An  examination  of  the  factors  involved  in  the  demonstrations  suggested 
several  ways  of  summarizing  the  data.  In  the  following  tables,  the 
studies  were  grouped  in  various  ways  in  accordance  with  the  presence  or 
absence  of  the  major  factors.  The  median  values  for  flight  syllabus 
reductions  and  substitution  ratios  were  computed  on  the  groupings. 
Descriptive  rather  than  inferential  statistics  were  used  in  these  compari- 
sons because  of  data  limitations.  The  skewed  raw  data  suggested  that 
comparing  median  values  was  the  most  appropriate  method  of  analysis. 

TRAINING  CATEGORY  COMPARISONS.  Pure  instrument  tasks  resulted  in  larger 
percent  saving  and  more  favorable  (i.e.,  smaller)  FSRs  than  contact/ 
familiarization  type  tasks  (table  8).  The  median  savings  for  demonstra- 
tions utilizing  both  types  of  training  fell  between  the  values  for 
instrument  and  contact/familiarization  tasks. 
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TABLE  8.  SUMMARY  OF  SUBSTITUTION  DATA  FOR  TRAINING  CATEGORIES 


Categories 

Number  of 
Comparison 
Demonstrations 

Median 

Percent  Flight 
Syllabus  Reduction 

Median  Flight 
Substitution 
Ratio 

Contact/ 

Familiarization 

4 

18 

1.8 

Pure  Instruments 

6 

50 

0.6 

Both 

15 

39 

2.1 

USER  CLASS  COMPARISONS.  Commercial  airlines  were  generally  able  to 
achieve  more  effective  substitution  than  other  simulator  users.  The 
comparisons  in  table  9 show  that  commercial  airline  median  syllabus 
reduction  was  significantly  better  than  that  of  other  users  (93  percent 
vs.  30  percent).  Their  FSRs  were  identical  to  noncommercial  users.  The 
similarity  in  FSRs  may  be  attributed  to  differences  in  transfer  efficiency 
decrements  (Povenmire  and  Roscoe,  1973).  That  is,  as  time  in  the  simulator 
increases,  the  amount  of  expected  transfer  per  hour  diminishes.  Hence, 
much  of  the  extensive  simulator  time  utilized  in  commercial  settings  is 
spent  at  asymptotic  levels,  producing  large  syllabus  reductions  but  with 
progressively  less  impact  on  FSRs. 


TABLE 

9.  SUMMARY  OF 

USER  CLASS  SUBSTITUTION 

DATA 

User 

Classes 

Number  of 
Comparison 
Demonstrations 

Median 

Percent  Flight 
Syllabus  Reduction 

Median  Flight 
Substitution 
Ratio 

Commercial 

3 

93 

1.1 

Noncommercial 

22 

30 

1.1 

TYPE  OF  AIRCRAFT.  For  the  three  major  military  aircraft  communities 
(helicopters,  jets,  and  transports),  it  was  found  that  simulators  were 
most  effectively  and  efficiently  used  for  helicopter  training.  Simulators 
were  less  successfully  utilized  in  jet  fighter-type  training  and  least 
successfully  employed  for  military  transport- type  programs.  These  results 
are  depicted  in  table  10.  This  is  unexpected  in  light  of  the  high  degree 
of  success  which  the  commercial  users  have  achieved  with  simulators  for 
similar  transport-type  airframes  (see  table  9). 
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TABLE  10.  SUMMARY  OF  SUBSTITUTION  DATA  BY  TYPE  OF  AIRCRAFT 


Military 

Aircraft 

Types 

Number  of 
Comparison 
Demonstrations 

Median 

Percent  Flight 
Syllabus  Reduction 

Median  Flight 
Substitution 
Ratio 

Helicopters 

5 

41 

0.8 

Jets 

6 

35 

1.3 

Transports 

7 

22 

2.3 

STUDENT  EXPERIENCE  COMPARISONS.  Data  in  table  11  indicate  a larger 
percent  syllabus  reduction  for  recent  graduate  vs.  undergraduate  programs. 
Median  syllabi  reductions  were  36  vs.  24  percent,  respectively.  However, 
the  rate  of  substitution  was  similar.  Airline  programs  were  excluded 
from  consideration  here  because  of  the  extremely  high  experience  levels  of 
their  pilots. 


TABLE  11.  SUMMARY  OF  SUBSTITUTION  DATA  FOR  STUDENT  EXPERIENCE  LEVELS 


Number  of 

Median 

Median  Flight 

Experience 

Comparison 

Percent  Flight 

Substitution 

Level 

Demonstrations 

Syllabus  Reduction 

Ratio 

Graduate 

11 

36 

0.9 

Undergraduate 

11 

24 

1.0 

Graduate  - Refers  to  demonstrations  involving  designated  military  pilots 
with  one  exception,  licensed  general  aviation  pilots  in  an 
instrument  training  course. 

Undergraduate  - Refers  to  eight  military  UPT  programs  and  three  general 
aviation  student  training  programs. 


SIMULATOR  CAPABILITY  COMPARISONS.  For  the  sake  of  clarity,  the  visual 
and  motion  parameters  of  simulator  capability  are  treated  separately  in 
this  section. 

Visual  System  Comparisons.  The  comparisons  in  table  12  provide  interesting 
results  in  that  devices  with  visual  systems  achieved  greater  syllabus 
reductions  than  those  devices  without  such  systems.  However,  the  median 
FSR  was  poorer  for  devices  equipped  with  visuals  (2.2  vs.  0.9).  Most 
studies  generally  show  that  visual  systems  are  relatively  inefficient  for 
training  contact  tasks. 
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TABLE  12.  SUMMARY  OF  SUBSTITUTION  DATA  FOR  VISUAL  SYSTEMS 


Visual 

Number  of 

Median 

Median  Flight 

Systems 

Comparison 

Percent  Flight 

Substitution 

Utilized 

Demonstrations 

Syllabus  Reduction 

Ratio 

Yes 

8 

48 

2.2 

No 

17 

36 

0.9 

Motion  System  Comparisons.  The  results  in  table  13  indicate  that  devices 
which  employed  high-fidelity  motion  platforms  achieved  much  more  effective 
syllabus  reduction  than  those  devices  which  did  not  employ  such  systems. 
Median  syllabi  reductions  were  42  percent  for  motion  equipped  devices  vs. 
16  percent  for  devices  which  lacked  motion.  The  FSRs  of  1.0  and  1.2  also 
imply  the  superiority  of  motion  equipped  devices. 


TABLE  13.  SUMMARY  OF  SUBSTITUTION  DATA  FOR  MOTION  SYSTEMS 


Motion 

Number  of 

Median 

Median  Flight 

Systems 

Comparison 

Percent  Flight 

Substitution 

Utilized 

Demonstrations 

Syllabus  Reduction 

Ratio 

Yes 

20 

42 

1.0 

No 

5 

16 

1.2 

CURRICULUM  FEATURE  COMPARISONS.  Comparisons  of  the  impact  of  special 
syllabi  and  part-task  training  in  conjunction  with  simulator  training  are 
treated  separately  in  this  section. 

Special  Syllabus  Comparisons.  The  development  of  a special  syllabus 
tailored  to  the  capabilities  of  a device  is  generally  necessary  to 
maximize  the  utility  of  the  simulator.  Table  14  summarizes  the  substitu- 
tion data  for  demonstrations  which  employed  such  special  syllabi.  For 
purposes  of  this  summary,  demonstrations  which  restricted  the  use  of 
self-pacing  to  the  new  simulator  group  are  also  treated  as  instances  of 
a special  syllabus.  Significant  reductions  were  achieved  in  such  special 
syllabus  programs  (40  vs.  16  percent).  Similarly,  the  median  FSR 
favors  the  special  syllabus  group  (1.0  vs.  2.3). 
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TABLE  14.  SUMMARY  OF  COMPARISONS  OF  SUBSTITUTION 


DATA  FOR  SPECIAL 

SYLLABI 

New  Simulator 
Groups  Given 
Special  Training 
Syllabus 

Number  of 
Comparison 
Demonstrations 

Median 

Percent  Flight 
Syllabus  Reduction 

Median  Flight 
Substitution 
Ratio 

Yes 

16 

40 

1.0 

No 

9 

16 

2.3 

Part-Task  Trainer  Comparisons.  A well -conceived  and  systematic  approach 
to  training  implies  the  use  of  part- task  trainers  in  many  situations.  The 
hours  spent  in  these  devices  were  not  included  in  the  simulator  time 
data  used  in  this  report.  Table  15  reveals  that  substitution  figures 
were  better  for  demonstrations  in  which  part-task  training  was  differen- 
tially used  in  conjunction  with  the  new  simulator  group  (but  not  with 
the  comparison  group).  The  median  syllabi  reductions  were  41  vs.  30  percent. 
Likewise,  their  median  FSRs  were  better  with  part-task  training  (0.9  vs. 

1.7). 


TABLE  15.  SUMMARY  OF  COMPARISONS  OF  SUBSTITUTION 
DATA  FOR  PART-TASK  TRAINING 


Part-Task 
Trainers  Also 

Number  of 

Median 

Median  Flight 

Used  By  New 

Comparison 

Percent  Flight 

Substitution 

Simulator  Groups 

Demonstrations 

Syllabus  Reduction 

Ratio 

Yes 

9 

41 

0.9 

No 

16 

30 

1 .7 
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SECTION  V 
SUMMARY 

This  report  summarizes  25  representative  demonstrations  of  simulator 
substitution  practices  conducted  in  the  past  decade.  The  information 
obtained  from  the  original  documents  was  of  sufficient  detail  to  allow 
computation  of  standard  substitution  indices  and  interdemonstration 
comparisons.  While  many  comparisons  could  have  been  made  of  the  data 
obtained  from  the  demonstrations,  the  following  represent  a condensation 
of  the  most  important  observations. 

Higher  flight  syllabus  reductions  and  better  flight  substitution 
ratios  were  attained  in  instrument  tasks  than  in  contact-type 
tasks. 

Commercial  airlines  have  achieved  far  better  flight  syllabus 
reductions  than  general  aviation  or  the  military. 

Simulators  in  military  programs  were  most  effectively  and 
efficiently  utilized  for  helicopter  training.  Jet  programs 
were  less  successful  and  transport  programs  were  least  successful. 

Graduate  programs  achieved  greater  flight  syllabus  reductions 
and  similar  flight  substitution  ratios  in  comparison  with 
undergraduate  programs. 

Simulation  devices  equipped  with  visual  systems  achieved  much 
greater  flight  syllabus  reductions  but  were  less  efficient  in 
terms  of  flight  substitution  ratios  than  devices  which  lacked 
such  systems.  That  is,  they  saved  greater  amounts  of  flight 
time  than  devices  without  such  systems,  but  they  required  more 
hours  of  simulator  time  per  flight  hour  saved. 

Devices  having  high-fidelity  motion  systems  achieved  much  larger 
flight  syllabus  reductions  than  devices  without  such  systems. 

Greater  flight  syllabus  reductions  and  more  efficient  flight 
substitution  ratios  were  achieved  when  special  syllabi  tailored 
to  the  simulator  were  used. 

Part-task  trainers  utilized  in  conjunction  with  new  simulators 
resulted  in  higher  flight  syllabus  reductions  and  better  flight 
substitution  ratios. 

In  substance,  however,  the  demonstrations  cited  indicate  substantial 
ambiguity  surrounding  these  practices  and  various  shortcomings  can  be 
identified.  The  more  telling  of  these  are  outlined  below. 
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DATA  QUALITY 

Substitution  data  and  old  syllabi  are  typically  not  retained 
by  operational  units.  The  lack  of  this  historic  data  prohibits 
before-and-after  comparisons  of  simulator  programs. 

. Syllabus  hours  may  be  inaccurately  reported  or  reasons  for 

syllabus  revisions  incorrectly  attributed  to  simulator  utilization. 

Data  often  lack  specificity  in  defining  training  tasks  and 
reporting  results.  The  data  are  further  confounded  by  simul- 
taneous changes  of  device,  instructional  strategy,  or  curriculum 
preventing  the  identification  of  a specific  cause  for  changes 
in  training  hours. 

Contradictions  in  data  involving  similar  operational  contexts 
may  produce  dissimilar  results  because  of  differential  emphasis 
on  flight  hour  reductions  vs.  improvements  in  quality  of 
training. 

METHODOLOGICAL  PROBLEMS 

The  student  performance  criteria  are  defined  in  subjective, 
ambiguous  nonstandard  terms.  Simulator  effectiveness  is 
also  often  validated  against  poorly  defined,  subjective,  norm- 
referenced  criteria;  e.g.,  "passed  flight  check." 

Task  or  training  analyses  which  are  necessary  but  not  sufficient 
to  establish  the  validity  of  training  requirements  are  not 
always  performed  prior  to  conducting  the  demonstration.  Such 
training  analyses  eliminate  irrelevant  variables  and  simplify 
the  assignment  problem  of  tasks  to  methods. 

The  demonstrations  are  characterized  by  the  lack  of  a programmatic 
approach.  In  essence,  each  demonstration  provided  a single 
"point  estimate"  of  all  the  feasible  simulator  hour/flight 
hour  combinations. 

DATA  INTERPRETATION  PROBLEMS 

A single  substitution  standard  does  not  take  cognizance  of 
individual  unit  needs  and  could  if  applied  universally  impose 
disproportionate  hardships  on  some  units. 

Generalization  of  findings  from  one  type  of  operational  setting 
to  another  should  be  accomplished  cautiously  and  with  full 
knowledge  of  situational  differences. 
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The  data  apply  primarily  to  generalized  aviator  training  such 
as  familiarization,  basic  and  radio  instruments,  and  airways 
navigation.  They  cannot  always  be  precisely  extrapolated 
to  tactical  training  tasks. 

POST  NOTE 

This  study  was  undertaken  with  the  expectation  that  an  analysis  of 
current  simulator  substitution  practices  would  yield  insights  for  selecting 
substitution  ratios  for  simulator  and  in-flight  training  time.  Instead, 
a congeries  of  problems  associated  with  current  practices  limited  the 
scope  of  the  original  endeavor  and  precluded  any  attempts  to  derive 
rigorous  prescriptions  for  users.  The  inability  to  develop  effective 
guidelines  for  substitution  practices  is  particularly  troublesome  in  view 
of  the  significant  costs  involved  in  these  decisions. 

At  present,  not  much  power  can  be  marshaled  to  provide  satisfactory 
guidelines  for  employing  simulator  training  as  a surrogate  for  in-flight 
training.  However,  the  role  of  simulation  in  flight  training  is  receiving 
considerable  attention  today,  generated  not  only  by  an  awareness  of 
increasing  training  simulator  capabilities  but  by  economic  pressures  as 
well.  Systematic  efforts  to  put  the  substitution  issue  in  perspective 
are  underway. 

Several  major  programs  concerned  with  optimizing  simulator  utilization 
in  flight  training  are  currently  underway.  The  TAEG  is  continuing  its  work 
of  efficiently  integrating  a new  state-of-the-art  flight  simulator  into  the 
ongoing  training  program  of  a Navy  P-3  Replacement  Squadron.  A study 
just  completed  determined  the  effectiveness  of  Device  2F87F  as  a sub- 
stitute for  the  earlier  generation  Device  2F69D  in  combination  with  the 
P-3  aircraft  in  training  replacement  patrol  plane  pilots  (Browning, 

Ryan,  Scott,  and  Smode,  1977).  Additional  study  will  examine  the  contri- 
butions of  the  motion  and  visual  systems  to  performance,  establish  a 
standardized  performance  assessment  capability  for  the  2F87F,  and  initiate 
an  effort  to  implement  an  automated  performance  measurement  system  in 
the  simulator. 

The  U.S.  Air  Force  has  an  extensive  program  of  research  utilizing 
the  ASUPT  at  Williams  Air  Force  Base,  Arizona.  Current  research  is 
examining  the  contribution  of  large  amplitude  motion,  the  G-seat,  the 
G-suit,  and  wide  angle  visual  simulation  to  performance. 

The  Army  Research  Institute  for  the  Behavioral  Sciences  utilizing 
the  Synthetic  Flight  Training  System  is  continuing  to  establish  optimal 
substitution  practices  for  specific  types  of  helicopters. 

To  deal  with  the  data  resulting  from  these  and  other  current  studies, 
a center  for  compiling  and  organizing  simulator  substitution  data  should 
be  established.  Its  purpose  would  be  to  catalog  all  study  efforts  and 
provide  a comprehensive  data  base  for  the  future. 
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